DESCRIPTION 



FEATURE CHANGED IMAGE GENERATING METHOD, FEATURE 
CHANGED IMAGE GENERATING APPARATUS AND FEATURE CHANGED 
5 IMAGE GENERATING PROGRAM 



Technical Field 

The present invention relates to a technique for 
changing a part of a feature of an image or adding other 

10 features to the image, so as to generate a new image. In 
particular, the present invention relates to a feature 
changed image generating method, a feature changed image 
generating apparatus and a feature changed image 
generating program, in which a new image of a face of a 

15 person is generated by adding a feature caused by aging 
to an image of the face of the person. 



Background Art 

A new image added with a certain feature while 
20 keeping original features has been often generated by 

adding the certain feature to an image. Typical examples 
include an image of an aged face of one and the same person, 
who has lost his or her young features caused by aging but 
has had aged features. An image of an aged face of one 
25 and the same person has been generated by eliminating a 
young feature from an image of a young face of a person 
while adding an aged feature thereto. 
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Examples of a method for generating an image of 
an aged face based on an image of a young face include a 
method for drawing an aged feature such as a crease in an 
image of a young face by using computer graphics 
5 (abbreviated as XX CG") . 

In this case, the aged feature such as the crease 
has depended upon an outline of a face. As a consequence, 
a manual work or a semi-automatic processing has been 
needed to apply "naturalness' 7 to a facial image to be 

10 generated. Here, there have been some features which are 
difficult to be drawn per se. Unlike the crease produced 
as a relatively clear feature, an aging level around an 
eye or a skin clearness of a face has variously depended 
upon a person. Therefore, it has been difficult to 

15 determine as to how such a feature is drawn in an image. 

U.S. Patent No. 6, 556, 196 Bl discloses an image 
processing method capable of adding an unclear feature to 
an image. In this image processing method, an aged feature 
can be clearly added to an image by the use of a 

20 three-dimensional model. More specifically, a general 
model (i.e., a prototype) of a deformable image of the face 
is generated based on three-dimensional facial data stored 
in a database. An inquiry facial image is stuck to the 
generated model. The model is degenerated by the use of 

25 a modeler in order to add changes in feature including an 
aged change. With this method, the same aged feature 
appears at the same portion even in the case where an image 
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of the face of anyone is processed since the previously 
prepared prototype is utilized. As a result, an unnatural 
aged feature may appear on the facial image. 

Japanese Laid-Open Patent Application JP- 
5 P2003-44866A discloses an image processing method capable 
of generating a target image based on a single specific 
image. In this method, an image of an exaggerated face 
is generated by extrapolation based on an image of a current 
face of a specific person and an image of an average face 

10 suitable for a current age. And then, an image of the face 
of the person having a target age is generated by 
interpolation based on an image of an average face having 
an age approximate to the target age and the image of the 
exaggerated face. With this method, no difference in aged 

15 change caused by variations among individuals in outlines 
of faces is taken into consideration since the image of 
the average face is used. Consequently, an unnatural aged 
feature may appear on the facial image. 

Japanese Laid-Open Patent Application JP-A- 

20 Heisei, 6-333005 discloses a facial image generating 

apparatus includes parts pattern storing means, facial 
feature data storing means, designating means and facial 
image generating means. The parts pattern storing means 
stores therein respective parts patterns for parts, which 

25 represent facial images. The facial feature data storing 
means stores therein facial feature data corresponding to 
ages. When the designating means specifies data relevant 
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to an age, the facial image generating means reads facial 
feature data in accordance with the specified data from 
the facial feature data storing means. And then, the 
facial image generating means reads the corresponding 
5 parts pattern of each of the parts from the parts pattern 
storing means in accordance with the facial feature data. 
In this manner, the facial image genera t ing means combines 
the parts patterns, so as to generate a facial image, 
Japanese Laid-Open Patent Application JP-A- 

10 Heisei, 10-289320 discloses a technique for speeding up 
calculation of a candidate category set in pattern 
recognition, A candidate table contained in table storage 
means holds therein mapping, in which a value of a reference 
feature vector calculated from a feature vector of a 

15 pattern is used as an input while the candidate category 
set is used as an output. Candidate category calculating 
means calculates a candidate category set corresponding 
to the value of the given reference feature vector based 
on the mapping of the candidate table, 

20 Japanese Laid-Open Patent Application JP- 

P2002-123837A discloses a facial expression transforming 
method comprising the steps of; (1) defining a code book 
storing data defining a first facial expression set of a 
first person; (2) preparing data defining a second facial 

25 expression set, which gives a training facial expression 
set of a second person different from the first person; 
(3) leading a transformation function out of the training 
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facial expression set and a corresponding facial 
expression included in the first facial expression set; 
and (4) applying the transformation function to the first 
facial expression set, so as to obtain a synthetic facial 
5 expression set. 

Japanese Laid-Open Patent Application JP- 
P2003-69846A discloses an image correcting program for 
automatically carrying out a proper image correction. The 
image processing program includes a correction processing 

10 pre-stage section, a statistic information calculation 
section and a correction processing post-stage section. 
The correction processing pre-stage section carries out 
correction of a range or a tone with respect to an input 
image. The statistic information calculation section 

15 produces a color saturation reference value and a contour 
reference value as data representing preferences of an 
operator by using an output from the correction processing 
pre-stage section and a manually corrected image. The 
correction processing post-stage section carries out a 

20 color saturation correction processing by the use of the 
color saturation reference value, and further, carries out 
a contour emphasis processing by the use of the contour 
reference value. 

25 Disclosure of Invention 

An object of the present invention is to provide 
a feature changed image generating method, a feature 
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changed image generating apparatus and a feature changed 
image generating program, in which other features can be 
added to an original image with a natural impression while 
keeping principal features of the original image. 
5 Another object of the present invention is to 

provide the feature changed image generating method, the 
feature changed image generating apparatus and the feature 
changed image generating program, in which an image of an 
aged face can be generated in consideration of variations 

10 among individuals, 

A further object of the present invention is to 
provide the feature changed image generating method, the 
feature changed image generating apparatus and the feature 
changed image generating program, in which a general aging 

15 change per age can be added to an input facial image, 

A still further object of the present invention 
is to provide the feature changed image generating method, 
the feature changed image generating apparatus and the 
feature changed image generating program, in which a 

20 distribution ratio can be adjusted when an aging change 
and an individual feature are added to an input facial 
image . 

According to an aspect of the present invention, 
a feature changed image generating method for generating 
25 a new image from an input image includes: (A) a step of 
providing a database in which a plurality of data, which 
are relating to a plurality of images respectively, are 
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classified into a plurality of categories; (B)a step of 
determining an image which is most similar to the input 
image as a selected image based on a data belonging to a 
specified category specified from the plurality of 
5 categories; and (C)a step of merging the selected image 
and the input image . 

At the step (A) , a database in which the plurality 
of images are classified into the plurality of categories 
is optionally provided. In this case, at the step (B), 

10 an image which is most similar to the input image among 
images belonging to the specified category is selected as 
the selected image. 

At the step (A) , a database in which 
constituent components of the plurality of images are 

15 classified into the plurality of categories is optionally 
provided. In this case, the step (B) includes: (bl) 
determining a determined combination of the constituent 
components by which an image which is most similar to the 
input image is obt a ined by us ing the constituent components 

20 belonging to the specified category; and (b2) generating 
an image which is most similar to the input image as the 
selected image based on the determined combination. 

At the step (A) , a database in which the plurality 
of images are classified into the plurality of categories 

25 is provided, and each of the plurality of categories 

includes a plurality of images which are gradual variations 
of an identical object on an attribute (for example, the 
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age), optionally. In this case, the step (B) includes: 
(bl) selecting an image which is most similar to the input 
image among images belonging to a category included in the 
plurality of categories and corresponding to an attribute 
5 of the input image as a similar image; and (b2) determining 
an image relating to a same object with the similar image 
as the selected image from images belonging to the 
specified category. 

At the step (A), a database in which constituent 

10 components of the plurality of images are classified into 
the plurality of categories is provided optionally, and 
each of the plurality of categories includes constituent 
components of a plurality of images which are gradual 
variations of an identical object on an attribute. In this 

15 case, the step (B) includes: (bl) selecting a selected 
combination of the constituent components by which an image 
which is most similar to the input image is obtained, by 
using the constituent components belonging to a category 
included in the plurality of categories and corresponding 

20 to an attribute of the input image; (b2) converting 
component coefficients corresponding to the selected 
combination into converted coefficients which are 
component coefficients corresponding to the specified 
category; and (b3) generating the selected image by using 

25 the converted coefficients and the constituent components 
belonging to the specified category. 

In the feature changed image generating method, 
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each of the plurality of images can be a face image of a 
person. Also, the plurality of categories can be 
categorized based on an age. 

When an age of a person on the input image is lower 
5 than an age specified by a user, a category included in 
the plurality of categories and corresponding to an age 
higher than the specified age can be selected as the 
specified category 

When an age of a person in the input image is higher 

10 than an age specified by a user, a category included in 
the plurality of categories and corresponding to an age 
lower than the specified age can be selected as the 
specified category 

In an another aspect of the present invention, 

15 a feature change applying method for gradually applying 
a feature change to an input image includes: (A) a step 
of providing a database in which constituent components 
of a plurality of images are classified into a plurality 
of categories, wherein each of the plurality of categories 

20 includes constituent components of a plurality of images 
which are gradual variations of an identical object on an 
attribute; (B) a step of selecting a selected combination 
of the constituent components by which an image which is 
most similar to the input image is obtained, by using the 

25 constituent components belonging to a category included 
in the plurality of categories and corresponding to an 
attribute of the input image; and (C) a step of converting 
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component coefficients corresponding to the selected 
combination into converted coefficients which are 
component coefficients corresponding to the specified 
category . 

5 In this feature change applying method, each of 

the plurality of images is a face image of a person, and 
the plurality of categories are categorized by based on 
an age . 

In yet another aspect of the present invention, 

10 the feature changed image generating device is an apparatus 
for realizing the above mentioned feature changed image 
generating method, and includes constituent elements 
realizing each of the above mentioned steps. The feature 
changed image generating apparatus has a storing unit, an 

15 image determining unit and a merging unit. The above 

mentioned database is built on the storing unit. The image 
determining unit executes the step (B) . The merging unit 
executes the step (C) . 

In yet another aspect of the present invention, 

20 the feature change applying apparatus is an apparatus for 
realizing the above mentioned feature change applying 
method, and includes constituent elements realizing each 
of the above mentioned steps. The feature change applying 
apparatus has a storing unit and a component coefficient 

25 converting unit. The above mentioned database is built 
on the storing unit. The component coefficient converting 
unit executes the step (B) and step (C) . 
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In yet another object of the present invention, 
the feature changed image generating program and the 
feature change applying program are the programs for 
realizing the above mentioned feature changed image 
5 generating method and the feature change applying method 
respectively. the feature changed image generating 
program and the feature change applying program 
respectively cause a computer to execute each of the above 
mentioned steps. 

10 According to the present invention, the image 

most similar to the input image is selected, and then, the 
input image and the selected image are merged with each 
other. As a consequence, other features can be added to 
the input image while keeping the original features of the 

15 input image. Furthermore, the input image is merged with 
the most similar image, so that the other features can be 
added to the input image with the natural impression. 



Brief Description of Drawings 

20 FIG. 1 is a block diagram illustrating a feature 

changed image generating apparatus in a first embodiment; 

FIG. 2 is a flowchart illustrating a feature 
changed image generating method in the first embodiment; 

FIG. 3 is a diagram illustrating a processing 
25 example, in which a maximum score image and an input image 
are linearly merged with each other; 

FIG. 4 is a block diagram illustrating a feature 
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changed image generating apparatus in a second embodiment; 

FIG. 5 is a flowchart illustrating a feature 
changed image generating method in the second embodiment; 

FIG. 6 is a block diagram illustrating a 
5 modification in the second embodiment; 

FIG. 7 is a flowchart illustrating a feature 
changed image generating method in the modification in the 
second embodiment ; 

FIG. 8 is a block diagram illustrating a feature 
10 changed image generating apparatus in a third embodiment; 

FIG. 9 is a flowchart illustrating a feature 
changed image generating method in the third embodiment; 

FIG. 10 is a block diagram illustrating a feature 
changed image generating apparatus in a fourth embodiment; 
15 FIG. 11 is a flowchart illustrating a feature 

changed image generating method in the fourth embodiment; 

FIG. 12 is a block diagram illustrating a feature 
changed image generating apparatus in a fifth embodiment; 

FIG. 13 is a flowchart illustrating a feature 
20 changed image generating method in the fifth embodiment; 

FIG. 14 is a block diagram illustrating a 
modification in the fifth embodiment; and 

FIG. 15 is a flowchart illustrating a feature 
changed image generating method in the modification in the 
25 fifth embodiment. 



Best Mode for Carrying Out the Invention 
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(First Embodiment ) 

A description will be given below of a first 
embodiment according to the present invention referring 
to the attached drawings. Here, explanation will be made 
5 on an example in which an aging change is added to the image 
of the face of the person (i.e., a facial image) . 

FIG. 1 is a block diagram illustrating a 
constitutional example of a feature changed image 
generating apparatus according to the present invention. 

10 As shown in FIG. 1, the feature changed image generating 
apparatus includes an image accumulating unit 101 serving 
as a database, a matching unit 102 for matching images and 
a merging unit 103 for merging images. The image 
accumulating unit 101 is implemented by, for example, a 

15 magnetic disk device. The matching unit 102 and the 
merging unit 103 are implemented by, for example, an 
arithmetic processor in a computer and a program executed 
by the arithmetic processor, respectively. Incidentally, 
in the present embodiment, a storing unit for storing 

20 information on a plurality of images corresponds to the 
image accumulating unit 101. In addition, an image 
determining unit for determining an image most similar to 
the input image corresponds to the matching unit 102. 

The image accumulating unit 101 serves as a 

25 database, in which numerous facial images are accumulated. 
In the image accumulating unit 101, the numerous facial 
images are classified into categories 11^ (i.e., a first 
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category), 11^ (i.e., an i-th category), lll n 

(i.e., an n-th category) according to age or sex (i.e., 
an attribute) . The categories to lll n are classified 

according to age or sex: for example, "a male in teens", 
5 "a female in twenties" and the like. In the case where 
the categories to lll n are comprehensively expressed 

or any one of the categories is expressed, they will be 
simply referred to as "category 111" hereinafter. 

FIG. 2 is a flowchart illustrating a feature 

10 changed image generating method by the feature changed 
image generating apparatus shown in FIG. 1. When a user 
specifies the input category 11 (age or sex) , the matching 
unit 102 selects a category 111 corresponding to the 
specified input category 11 from the image accumulating 

15 unit 101 (step Sll). For example, in the case where a 
facial image of amale in teens is input to generate a facial 
image assuming the male in twenties, the user inputs 
"twenties" and "male" as the input category 11. Then, the 
matching unit 102 selects the category 111 of "a male in 

20 twenties". Here, the user may input not ages but an age 
per se. In this case, the matching unit 102 selects the 
category 111 corresponding to the ages including the input 
age . 

Furthermore, the matching unit 102 may not always 
25 select the category 111 of an age bracket specified by the 
input category 11 but select another category 111. For 
example, the merging unit 103 may merge facial images while 
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regarding an aged change as a linear change, as described 
later. In this case, the matching unit 102 receives not 
only information on the age specified by the input category 

11 (i.e., a target age) but also information on an age of 
5 a person in the input image 12 (i.e., an input person's 

age) . If the target age is greater than the input person's 
age, the matching unit 102 may select the category 111 of 
an age bracket much greater than the target age. In 
contrast, if the target age is less than the input person's 

10 age, the matching unit 102 may select the category 111 of 
an age bracket much less than the target age. 

For example, in the case where the age of the 
person in the input image 12 ranges within twenties while 
the age bracket specified by the input category 11 ranges 

15 within thirties, the matching unit 102 may select the 

category 111 of forties. In this manner, a facial image 
in thirties can be generated by linearly merging the input 
image 12 in twenties with a facial image in forties. To 
the contrary, in the case where the age of the person in 

20 the input image 12 ranges within forties while the age 
bracket specified by the input category 11 ranges within 
thirties, the matching unit 102 may select the category 
111 of twenties. In this manner, the facial image in 
thirties can be generated by linearly merging the input 

25 image 12 in forties with the facial image in twenties. 

The matching unit 102 receives the input image 

12 (step S12). Subsequently, the matching unit 102 
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matches the input image 12 with a group including facial 
images belonging to the selected category (step S13) . The 
matching unit 102 performs the matching of the group 
including the facial images based on a general algorithm 
5 for use in a face recognizing processing. Specifically, 
the matching unit 102 compares facial features between the 
facial images included in the selected category 111 with 
the input image 12, thereby obtains the degree of 
similarity between each of the facial images and the input 

10 image 12, The facial features include the position or 
shape of the eye, the nose or the mouth or the entire facial 
contour. The obtained degree of similarity as a score 
corresponds to each of the facial images. 

The matching unit 102 selects a facial image 

15 having the highest score (i.e., the highest score image) 
among the group including the facial images belonging to 
the selected category as a facial image most similar to 
the face in the input image 12 (step S14) . In other words, 
the matching unit 102 can select a facial image most similar 

20 in principal parts of a face such as the shape of the eye, 
the mouth or the face to the input image 12 among the group 
including the facial images included in the selected 
category. Incidentally, in the case where it is 
determined such that the score becomes smaller as the 

25 degree of similarity becomes greater, the matching unit 
102 selects a facial image having the smallest score. The 
matching unit 102 outputs a facial image most similar to 



- 17 - 

the input image 12 as a selected facial image to the merging 
unit 103. 

The merging unit 103 merges the input image 12 
with the selected facial image, thereby generates a merged 
5 image 14 (step S15). The merging unit 103 outputs the 
generated merged image 14. 

The merging unit 103 merges the facial images with 
each other by, for example, "a linear merging method" . For 
example, the merging unit 103 normalizes the selected 

10 facial image such that the eye, the nose or the mouth (i.e., 
a feature) in the selected facial image is located at the 
same position as that of the input image 12, thereby 
generating a normalized facial image. Moreover, the 
merging unit 103 weighted-averages pixel data at a 

15 corresponding portion between the input image 12 and the 
normalized facial image, thereby generating the merged 
image 14. Here, the facial image merging method by the 
merging unit 103 is not limited to the linear merging 
me thod . 

20 FIG. 3 illustrates one example of a linear merging 

processing by the use of the selected facial image and the 
input image 12. Here, explanation will be made on an 
example in which a person of the input image 12 has an age 
in twenties, the age bracket specified by the input 

25 category 11 ranges within thirties, and the matching unit 
102 selects the category 111 corresponding to the forties. 

In FIG. 3, it is assumed that a first facial image 



- 18 - 

is the input image 12 and a second facial image is the 
selected facial image. With the use of a certain parameter 
a, a merging ratio of the first facial image to the second 
facial image is expressed by a: (1-cx) . Here, the parameter 
5 a is a value of 0 or more and 1 or less. In this example, 
since the age of the person of the merged image 14 is 
required to fall within thirties, the parameter a is set 
to be 0.5: namely, the merging ratio is set to be 1:1. The 
merging unit 103 sets the parameter oc to 0.5, and thus, 

10 the merged image (i.e., the merged facial image) 14 is 
generated, as illustrated in FIG. 3, by taking a weighted 
average between the input image 12 and the normalized 
facial image . 

Incidentally, the merging unit 103 may merge the 

15 input image 12 and the selected facial image with each other 
while varying the merging ratio during the image merging 
processing. In this case, the merging ratio is stepwise 
adjusted, so that an aged change from the age of the person 
of the input image 12 to a specified age can be stepwise 

20 confirmed. 

In the present embodiment, the facial images have 
been classified into the plurality of categories according 
to age or sex, to be then stored in the image accumulating 
unit 101. Here, a facial image classifying method is not 

25 limited to the method in the present embodiment. For 

example, the age may be replaced with a group such as an 
idol group as the category creating criterion. In this 
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case, respective facial images of members of an idol group 
Al are stored in the first category, and further, 
respective facial images of members of an idol group A2 
are stored in a second category. And then, the facial 
5 image of a person most similar to a member in a specified 
idol group may be merged with an input facial image 12, 
thereby generating the merged image 14. In this manner, 
the present invention is applicable to an amusement. 

As described above, in the present embodiment, 

10 the specified facial image group is selected from the 
plurality of the classified facial image sets; the facial 
image most similar to the input image 12 is extracted from 
the selected facial image set; and the input image 12 is 
merged with the extracted facial image. As a consequence, 

15 other features can be added to the input image 12 while 
keeping the original features of the input image 12. 
Additionally, since the input image 12 is merged with the 
most similar facial image, other features can be added to 
the input image 12 in such a manner as to give the natural 

20 impression. Consequently, a feature of a secondary 

attribute can be added while keeping the principal feature 
of the original facial image, and further, the feature of 
the secondary attribute can be added to the facial image 
in such a manner as to give the natural impression. 

25 Specifically, the selected facial image having 

the principal parts of the face such as the shape of the 
eye, the mouth or the face most similar to those of the 
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input image 12 is merged with the input image 12. 
Therefore, the feature of the secondary attribute can be 
added to the facial image in such a manner as to give the 
natural impression while keeping the principal feature as 
5 an element for identifying the person. Here, the 

secondary attribute signifies an attribute such as the 
crease or a dimple which does not adversely influence on 
the identification of the person. 

Furthermore, in the present embodiment, since the 

10 matching unit 102 selects the aged facial image similar 
to the input image 12, an aged feature peculiar to the 
outline of the face of the person of the input image 12 
can be readily added to the input image 12. Moreover, the 
merged facial image can be readily generated without any 

15 necessity of consideration of the aged feature of each of 
the parts of the face such as the eye or the nose. 

Additionally, in the present embodiment, since 
the image accumulating unit 101 classifies the facial 
images according to the ages and stores them therein, the 

20 facial image can be generated by designating a specific 
age. In addition, it is possible to generate a facial 
image having not only the aged feature but also a younger 
feature . 

Moreover, in the present embodiment, since the 
25 merging ratio of the input image 12 to the selected facial 
image can be varied when the merging unit 103 performs the 
merging processing. The merging ratio during the merging 
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processing can be adjusted, so that the aged change from 
the input image 12 can be stepwise confirmed. Furthermore, 
since an existing recognition system can be utilized in 
the present embodiment, the system can be readily assembled 
5 or modified. 

(Second Embodiment ) 

Next, a description will be given below of a second 
embodiment according to the present invention referring 
to the attached drawings. FIG. 4 is a block diagram 

10 illustrating a constitutional example of the feature 
changed image generating apparatus in the second 
embodiment. As shown in FIG. 4, the feature changed image 
generating apparatus includes an image component 
accumulating unit 101b, a component analyzing unit 102b 

15 for analyzing a component of an image and the merging unit 
103 for merging images. The image component accumulating 
unit 101b is implemented by, for example, a magnetic disk 
device. The component analyzing unit 102b and the merging 
unit 103 are implemented by, for example, an arithmetic 

20 processor in the computer and the program executed by the 
arithmetic processor, respectively. 

Incidentally, in the present embodiment, the 
storing unit for storing information on a plurality of 
images corresponds to the image component accumulating 

25 unit 101b. In addition, an image determining unit for 
determining an image most similar to the input image 
corresponds to the component analyzing unit 102b. 
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The image component accumulating unit 101b serves 
as a database, in which information on a plurality of facial 
images is accumulated. The image component accumulating 
unit 101b stores not facial images per se but a plurality 
5 of constituent components obtained by analyzing 

components of the facial image. A component analysis is 
exemplified by the principal component analysis. 

Specifically, the plurality of facial images are 
classified into a plurality of categories according to age 

10 or sex. The constituent components obtained by analyzing 
the components of each of the facial images are stored in 
a manner corresponding to each of the categories in the 
image component accumulating unit 101b. For example, one 
vector can be obtained by arranging pixels of each of the 

15 facial images, and then, constituent components obtained 
by subjecting the vector to singular value decomposition 
are stored. As a result, in the image component 
accumulating unit 101b, the constituent components of each 
of the facial images are classified into categories 112! 

20 (i.e., the first category), 112 i (i.e., the i-th 

category), 112 n (i.e., the n-th category) according 

to age or sex. The categories 112 x to 112 n are classified 
according to age or sex: for example, "a male in teens", 
"a female in twenties" and the like. In the case where 

25 the categories 112! to 112 n are comprehensively expressed 
or any one of the categories is expressed, they will be 
simply referred to as "category 112" hereinafter. 
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FIG. 5 is a flowchart illustrating the feature 
changed image generating method by the feature changed 
image generating apparatus shown in FIG. 4. When a user 
specifies the input category 11 (age or sex) , the component 
5 analyzing unit 102b selects the category 112 corresponding 
to the specified input category 11 from the image component 
accumulating unit 101b (step S21). 

Incidentally, the component analyzing unit 102b 
may not always select the category 112 of an age bracket 

10 specified by the input category 11 but select another 

category 112 . For example, the merging unit 103 may merge 
facial images while regarding an aged change as a linear 
change, as described later. In this case, the component 
analyzing unit 102b receives not only information on the 

15 age specified by the input category 11 (i.e., a target age) 
but also information on an age of a person of the input 
image 12 (i.e., an input person's age) . If the target age 
is greater than the input person's age, the component 
analyzing unit 102b may select the category 112 of an age 

20 bracket much greater than the target age. In contrast, 
if the target age is less than the input person's age, the 
component analyzing unit 102b may select the category 112 
of an age bracket much less than the target age. 

For example, in the case where the age of the 

25 person of the input image ranges within twenties while the 
age bracket specified by the input category 11 ranges 
within thirties, the component analyzing unit 102b may 
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select the category 112 of forties. To the contrary, in 
the case where the age of the person of the input image 
ranges within forties while the age bracket specified by 
the input category 11 ranges within thirties, the component 
5 analyzing unit 102b may select the category 112 of 
twent ie s . 

The component analyzing unit 102b generates "a 
minimum deviation reconstructed image" as a facial image 
most similar to the input image 12 by the use of the 

10 constituent components stored in the image component 
accumulating unit 101b. In the present embodiment, 
processing for generating the facial image similar to the 
input image 12 by the use of the constituent components 
by the component analyzing unit 102b is regarded as the 

15 reconstruction of the input image 12. 

The component analyzing unit 102b reconstructs 
the input image 12 by the use of the constituent components 
corresponding to the selected category (step S23) upon 
receipt of the input image 12 (step S22). The component 

20 analyzing unit 102b reconstructs the input image 12 such 
that the deviation of a facial image to be generated with 
respect to the input image 12 becomes minimum. In other 
words, the component analyzing unit 102b carries out the 
reconstruction in such a manner as to maximize the degree 

25 of similarity of the facial image to be generated to the 
input image 1 2 . 

For example, in the case of the use of the linear 
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component analysis such as the principal component 
analysis, a facial image to be generated is expressed by 
Equation (1) , as described below. That is to say, a facial 
image I p to be generated is expressed as linear combination 
5 of principal components (i.e., constituent components) by 
using a coefficient c i (a real number) and a principal 
component P t obtained by the principal component analysis. 
Here, the principal component Pi is a vector of a real number 
having the same number of elements as the total number of 
10 pixels of a facial image in Equation (1) . 

I=cP+cP H he P •••(!) 

p 1 1 2 2 mm 

The component analyzing unit 102b determines a 
combination of the constituent components (specifically, 
a value of each of the coefficients) with a minimum 

15 deviation from a facial image I 0 input as the input image 
12 based on Equation (1) by using a constituent components 
in the selected category. Thereafter, the component 
analyzing unit 102b generates a facial image in accordance 
with the determined combination of the constituent 

20 components. And then, the component analyzing unit 102b 
outputs the generated facial image as the minimum deviation 
reconstructed image . 

The merging unit 103 merges the input image 12 
with the minimum deviation reconstructed image, thereby 

25 generating the merged image 14, and then, outputting the 
generated merged image 14 (step S24). The merging unit 
103 generates the facial image in the same method as that 



- 26 - 

in the first embodiment. 

As described above, according to the present 
embodiment, the specified constituent component set is 
selected from the plurality of classified constituent 
5 component sets; the minimum deviation reconstructed image 
most similar to the input image 12 is generated by the use 
of the selected constituent component set; and the minimum 
deviation reconstructed image is merged with the input 
image 12. As a consequence, other features can be added 

10 to the input image 12 while keeping the original features 
of the input image 12. Additionally, since the input image 
12 is merged with the most similar minimum deviation 
reconstructed image, other features can be added to the 
input image 12 in such a manner as to give the natural 

15 impression. Consequently, the feature of the secondary 
attribute can be added while keeping the principal feature 
of the original facial image, and further, the feature of 
the secondary attribute can be added to the facial image 
in such a manner as to give the natural impression. 

20 Specifically, the image having the principal 

parts of the face such as the shape of the eye, the mouth 
or the face most similar to the input image 12 can be 
generated by the reconstruction, and then, the image is 
merged with the input image 12. Therefore, the feature 

25 of the secondary attribute can be added to the facial image 
in such a manner as to give the natural impression while 
keeping the principal feature. Here, the secondary 
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attribute signifies an attribute such as the crease or the 
dimple which does not adversely influence on the 
identification of that person. 

Furthermore, in the present embodiment, since the 
5 component analyzing unit 102b reconstructs the aged facial 
image similar to the input image 12, an aged feature 
peculiar to the outline of the face of the person in the 
input image 12 can be readily added to the input image 12. 
Moreover, the merged facial image can be readily generated 
10 without any necessity of consideration of the aged feature 
of each of the parts of the face such as the eye or the 
no s e . 

Additionally, in the present embodiment, since 
the image component accumulating unit 101b classifies the 

15 constituent components according to the age and stores them 
therein, the facial image can be generated by designating 
a specific age. In addition, it is possible to generate 
a facial image having not only the aged feature but also 
a younger feature. 

20 On the other hand, in the case where the minimum 

deviation reconstructed image sufficiently similar to the 
input image 12 cannot be generated at one time, the 
reconstructing processing may be repeatedly performed. 
FIG. 6 is a block diagram illustrating a modification, in 

25 which the configuration of the feature changed image 

generating apparatus shown in FIG. 4 is partly modified. 
FIG. 7 is a flowchart illustrating the feature changed 
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image generating method in the feature changed image 
generating apparatus shown in FIG. 6. Here, the 
processing in steps S21 to S23 in FIG. 7 is the same as 
that in steps S21 to S23 in FIG. 5. 
5 In the modification shown in FIG. 6, in the case 

where the degree of similarity of the minimum deviation 
reconstructed image to the input image 12 is lower than 
a predetermined value (Yes in step S24a) , the merging unit 
103 merges the input image 12 and the minimum deviation 

10 reconstructed image with each other, and then, outputs a 
merged image to the component analyzing unit 102b (step 
S24b) . In other words, the merging unit 103 feeds back 
the merged image to the component analyzing unit 102b. 

Upon receipt of the merged image, the component 

15 analyzing unit 102b reconstructs the input image 12 in the 
same processing as that in step S23 based on the input 
merged image, and then, generates the minimum deviation 
reconstructed image again (step S25). The component 
analyzing unit 102b outputs the minimum deviation 

20 reconstructed image to the merging unit 103. The merging 
unit 103 merges the merged image, which has been recently 
fed back, with the minimum deviation reconstructed image, 
which has been input again from the component analyzing 
unit 102b, to generate another merged image 14, and 

25 thereafter, output it (step S26) . Incidentally, although 
only one feedback is shown in FIG. 7, the processing in 
steps S24a and S25 is performed again in the case where 
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the degree of similarity of the minimum deviation 
reconstructed image generated in step S25 to the input 
image 12 is still lower than the predetermined value. 

As described above, even if the input image 12 
5 and an image space contained in the category 112 are 

materially different from each other, the reconstructed 
image can be matched with the input image 12 by repeating 
the reconstructing processing. In other words, even if 
the degree of similarity to the input image 12 is absolutely 

10 low, it is possible to generate a reconstructed image 
having a relatively high similarity. 

In the present embodiment, the constituent 
components based on the facial image have been classified 
into the plurality of categories according to age or sex, 

15 to be then stored in the image component accumulating unit 
101b. Here, a facial image classifying method is not 
limited to the method in the present embodiment. For 
example, the age may be replaced with a group such as an 
idol group as the category creating criterion. In this 

20 case, respective facial images of members of the idol group 
Al are stored in the first category, and further, 
respective facial images of members of the idol group A2 
are stored in the second category. And then, the facial 
image of the person most similar to the member in the 

25 specified idol group may be merged with the input facial 
image 12, thereby generating the merged image 14. In this 
manner, the present invention is applicable to the 
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amusement . 
(Third Embodiment ) 

Next, a description will be given below of a third 
embodiment according to the present invention referring 
5 to the attached drawings. FIG. 8 is a block diagram 
illustrating a constitutional example of the feature 
changed image generating apparatus in the third embodiment. 
As illustrated in FIG. 8, the feature changed image 
generating apparatus includes an aging image accumulating 

10 unit 101c, a matching unit 102 for matching images and the 
merging unit 103 for merging images. The aging image 
accumulating unit 101c is implemented by, for example, a 
magnetic disk device. 

Incidentally, in the present embodiment, the 

15 storing unit for storing information on a plurality of 
images is equivalent to the aging image accumulating unit 
101c. The matching unit 102 and the merging unit 103 carry 
out the process similar to those of the first embodiment. 

The aging image accumulating unit 101c serves as 

20 a database, in which facial images having features changed 
with age are accumulated per age with respect to each of 
numerous persons (e.g., a person A to a person X) . 
Specifically, the aging image accumulating unit 101c 
classifies facial images of a certain person, gradually 

25 changed with age, into categories 113 1 (i.e., the first 
category) to 113 n (i.e., the n-th category) according to 
age, and then, stores them. In the case where the 
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categories 113 x to 113 n are comprehensively expressed or 
any one of the categories is expressed, they will be simply 
referred to as "category 113" hereinafter. 

FIG. 9 is a flowchart illustrating the feature 
5 changed image generating method by the feature changed 
image generating apparatus illustrated in FIG. 8. In the 
present embodiment, the matching unit 102 receives 
information designating an age 15 of the person of the input 
image 12 from the user (step S31) , and further, receives 

10 the input image 12 (step S32) . The matching unit 102 

selects one category 113 corresponding to the age 15 of 
the person among the plurality of categories 113 contained 
in the aging image accumulating unit 101c. The matching 
unit 102 matches all of facial images included in the 

15 selected category 113 with the input image 12. And then, 
the matching unit 102 determines a facial image having the 
maximum degree of similarity to the input image 12 among 
the facial images included in the selected category 113 
(step S33) . Here, it is assumed that a facial image of 

20 a person B is determined. Furthermore, it is assumed that 
a category 113* is selected in step S33. 

When the input category (i.e., a specified age) 
11 is specified by the user, the matching unit 102 selects 
a facial image of the same person as the person of the image 

25 determined in step S33 (in this case, the facial image of 
the person B) among facial images included in the category 
113 corresponding to the specified age (e.g., a category 
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113 n ) (step S34). Thereafter, the matching unit 102 
outputs the selected facial image as a selected facial 
image to the merging unit 103, The merging unit 103 merges 
the input image 12 with the selected facial image, thereby 
5 generating the merged image 14, and then, outputting the 
generated merged image 14 (step S35) . 

As described above, in the present embodiment, 
the facial image of the person having the specified age 
of the person of the image most similar to the input image 

10 12 is extracted; and the input image 12 is merged with the 
extracted facial image. As a consequence, other features 
can be added to the input image 12 while keeping the 
original feature of the input image 12. Additionally, 
other features can be added to the input image 12 in such 

15 a manner as to give the natural impression. Consequently, 
the feature of the secondary attribute can be added while 
keeping the principal feature of the original facial image, 
and further, the feature of the secondary attribute can 
be added to the facial image in such a manner as to give 

20 the natural impression. 

Furthermore, in the present embodiment, since the 
matching unit 102 selects the aged facial image of the 
person of the image similar to the input image 12, an aged 
feature peculiar to the outline of the face of the person 

25 in the input image 12 can be readily added to the input 
image 12 . Moreover, the merged facial image can be readily 
generated without any necessity of consideration of the 
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aged feature of each of the parts of the face such as the 
eye or the nose. 
(Fourth Embodiment ) 

Next, a description will be given below of a fourth 
5 embodiment according to the present invention referring 
to the attached drawings. FIG. 10 is a block diagram 
illustrating a constitutional example of the feature 
changed image generating apparatus in the fourth 
embodiment. As shown in FIG. 10, the feature changed image 

10 generating apparatus includes an aging image component 
accumulating unit lOld, the component analyzing unit 102b 
for analyzing a component of the image and a component 
coefficient conversing unit 104 for converting a component 
coefficient. The component analyzing unit 102b carries 

15 out the process similar to those of the second embodiment. 
The aging image component accumulating unit lOld is 
implemented by, for example, a magnetic disk device. The 
component coefficient conversing unit 104 is implemented 
by, for example, an arithmetic processor in the computer 

20 and the program executed by the arithmetic processor. 

Incidentally, in the present embodiment, the storing unit 
for storing information on a plurality of images 
corresponds to the aging image component accumulating unit 
lOld. 

25 The aging image component accumulating unit lOld 

serves as a database, in which information on a plurality 
of persons is accumulated. The aging image component 
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accumulating unit lOld stores not facial images per se but 
a plurality of constituent components obtained by 
analyzing components of the facial image. A component 
analysis is exemplified by the principal component 
5 analysis. Specifically, the plurality of facial images 
are classified into a plurality of categories according 
to age or sex. The constituent components obtained by 
analyzing the components of each of the facial images are 
stored in a manner corresponding to each of the categories 

10 in the aging image component accumulating unit lOld. 

Specifically, in the aging image component accumulating 
unit lOld, the constituent components are classified into 
categories 114 x (i.e., the first category) to 114 n (i.e., 
the n-th category) according to an age bracket of teens 

15 or twenties. In the case where the categories 114 x to 114 n 
are comprehensively expressed or any one of the categories 
is expressed, they will be simply referred to as "category 
114" hereinafter. Incidentally, an image of a face of one 
and the same person is contained in any two of the 

20 categories before the component analysis. 

The component coefficient conversing unit 104 
converts a coefficient at the time when the constituent 
components contained in each of the categories 114 are 
analyzed. The present embodiment exemplifies a case where 

25 a principal component analysis is used as the component 
analysis in the same manner as in the second embodiment. 
Explanation will be made on coefficient 
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conversion carried out by the component coefficient 
conversing unit 104. The two categories 114 for use in 
the component analysis are defined as a category A and a 
category B. In addition, principal components (i.e., 
5 constituent components) contained in the category A and 
the category B, respectively, are specified by Pi (wherein 
i is 1 to n) and Q ± (wherein i is 1 to m) , respectively. 
Furthermore, respective coefficients corresponding to the 
principal components Pi and Q i are denoted by c i (wherein 

10 i is 1 to n) and di (wherein i is 1 to m), respectively. 
A description will be given below of a case where the 
coefficient ci is converted into the coefficient di . 

Facial images before and after an aged change of 
one and the same person, generated by the use of the 

15 constituent components contained in the category A and the 
category B are specified by I p and J p , respectively. At 
this time, the facial images I p and J p are expressed by 
Equations (2) and (3), respectively. 

I =c P +c P H he P • • • (2) 

P 1 1 2 2 n n 

J =d Q +d Q +---+d Q —(3) 

2 0 P 1 1 2 2 mm 

As a result, the coefficient di can be obtained 
by linearly converting the coefficient Ci in accordance 
with Equation (4), as follows: 
[ Equat ion 1 ] 
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In Equation (4), a matrix A = {a^} is obtained 
by calculating a generalized inverse matrix. As a 
consequence, both of the category A and the category B in 
5 the categories 114 need commonly contain the constituent 
components of at least n or more of one and the same persons . 
An element ot ±1i in the matrix A is an inter-age conversion 
coefficient for converting the constituent components 
between ages . 

10 FIG. 11 is a flowchart illustrating the feature 

changed image generating method by the feature changed 
image generating apparatus illustrated in FIG. 10. When 
the user inputs the age 15 of the person of the input image 
12, the component analyzing unit 102b selects the category 

15 114 corresponding to an age bracket including the age 15 
of the person (step S41). The component analyzing unit 
102b reconstructs the input image 12 by the use of the 
constituent components contained in the selected category 
114 (step S43) upon receipt of the input image 12 (step 

20 S42) . The component analyzing unit 102b reconstructs the 
input image 12 such that the deviation of a facial image 
to be generated with respect to the input image 12 becomes 
minimum value. In other words, the component analyzing 
unit 102b carries out the reconstruction in such a manner 
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as to maximize the degree of similarity of the facial image 
to be generated to the input image 12. 

When an input category (i.e., the specified age) 
lib is input by the user, the component analyzing unit 102b 
5 selects the category 114 corresponding to the specified 
age (step S44) . The component coefficient conversing unit 
104 converts each of the coefficients at the time of the 
reconstruction into a coefficient in the category 114 
corresponding to the specified age (step S45) in accordance 

10 with Equation (4) . 

Then, the component analyzing unit 102b generates 
a minimum deviation reconstructed image 13b in accordance 
with Equation (3) by the use of the coefficient after the 
conversion and the constituent component contained in the 

15 category 114 corresponding to the specified age, and then, 
outputs it (step S46). 

As described above, in the present embodiment, 
the category 114 is configured such that any two of the 
categories contain the constituent components regarding 

20 the face of one and the same person. The input image 12 
is reconstructed by using the constituent components in 
the category 114 corresponding to the age 15 of the person. 
The coefficient at the time of the reconstruction is 
converted into the coefficient in the category 114 

25 corresponding to the specified age. Thereafter, the 

minimum deviation reconstructed image 13b is generated by 
using the coefficient after the conversion. Thus, it is 
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possible to obtain the image sufficiently expressing the 
side feature when the face of the input image 12 is changed 
wi th age . 
( Fi f th Embodiment ) 
5 Next, a description will be given below of a fifth 

embodiment according to the present invention referring 
to the attached drawings. FIG. 12 is a block diagram 
illustrating a constitutional example of the feature 
changed image generating apparatus in the fifth embodiment. 

10 The feature changed image generating apparatus 

illustrated in FIG. 12 includes the merging unit 103 for 
merging the input image and the minimum deviation 
reconstructed image with each other in addition to the 
configuration illustrated in the fourth embodiment. 

15 FIG. 13 is a flowchart illustrating the feature 

changed image generating method in the feature changed 
image generating apparatus illustrated in FIG. 12. Here, 
the processing in steps S41 to S46 in FIG. 13 is the same 
as that in steps S41 to S46 in FIG. 11. 

20 Like in the second embodiment, the merging unit 

(i.e., an image merging unit) 103 merges the input image 
12 with the minimum deviation reconstructed image upon 
receipt of the minimum deviation reconstructed image from 
the component analyzing unit 102b, thereby generating the 

25 merged image 14. And then, the merging unit 103 outputs 
the generated merged image 14 (step S57) . 

Here, in the case where the minimum deviation 
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reconstructed image sufficiently similar to the input 
image 12 cannot be generated, the reconstructing 
processing may be repeatedly performed. FIG. 14 is a block 
diagram illustrating a modification, in which the 
5 configuration of the feature changed image generating 
apparatus illustrated in FIG. 12 is partly modified. 
Moreover, FIG. 15 is a flowchart illustrating the feature 
changed image generating method in the feature changed 
image generating apparatus illustrated in FIG. 14. 

10 In the modification illustrated in FIG. 14, like 

in the modification illustrated in FIG. 6, in the case where 
the degree of similarity of the minimum deviation 
reconstructed image to the input image 12 is lower than 
a predetermined value (step S57a), the merging unit 103 

15 outputs an image obtained by merging the input image 12 
and the minimum deviation reconstructed image with each 
other, to the component analyzing unit 102b (step S57b) . 
In other words, the merging unit 103 feeds back the merged 
image to the component analyzing unit 102b. 

20 Upon receipt of the merged image, the component 

analyzing unit 102b reconstructs a facial image based on 
the input merged image (step S58). A component 
coefficient conversing unit 104 converts each of 
coefficients of the reconstructed facial images into a 

25 coefficient in a category corresponding to the specified 
age (step S59). The component analyzing unit 102b 
generates again the minimum deviation reconstructed image 
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by the use of the coefficients after the conversion and 
constituent components in the category corresponding to 
the specified age. The component analyzing unit 102b 
outputs the generated minimum deviation reconstructed 
5 image to the merging unit 103 (step S60) . 

The merging unit 103 merges the merged image, 
which has been lastly fed back, with the minimum deviation 
reconstructed image, which has been input again from the 
component analyzing unit 102b, to generate the merged image 

10 14, and thereafter, output it (step S61) . Incidentally, 
although only one feedback is illustrated in FIG. 15, the 
processing in steps S57b and thereafter is performed again 
in the case where the degree of similarity of the minimum 
deviation reconstructed image generated in step S60 to the 

15 input image 12 is still lower than the predetermined value. 

Even if the input image 12 and an image space 
contained in the category 114 are mat er i al ly different from 
each other, the reconstructed image can be matched with 
the input image 12 by repeating the reconstructing 

20 processing. 

Incidentally, the feature changed image 
generating apparatus, in which the facial image of the 
person is changed with age, has been mainly illustrated 
in the above-mentioned embodiments. However, the present 

25 invention is applicable to a case where the feature is added 
to the image other than the facial image in addition to 
the case where the feature is added to the facial image. 
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Furthermore, the feature changed image 
generating apparatus in the above-mentioned embodiments 
can be implemented by the computer. Specifically, 
programs for achieving the functions of the matching unit 
102, the merging unit 103, the component analyzing unit 
102b and the component coefficient conversing unit 104 may 
be provided, to be stored in a storing unit in the computer. 
An arithmetic processor in the computer executes 
processing in accordance with the programs, thus achieving 
the feature changed image generation in each of the 
embodiment s . 

Moreover, the present invention is applicable to 
generation of a montage changed with age. Even in the case 
where there is only a photograph of someone in youth, a 
facial image assuming an aged change can be generated. 
Furthermore, the present invention can be applied to a 
cellular mobile phone with a camera or an amusement 
application for use in the amusement arcade or the like. 



